1. Technical Field
The present invention relates to the field of testing software applied to hardware, and more specifically to a method and apparatus for provisioning and scheduling resources for testing in a data center resource.
2. Description of Related Art
Commercial software development typically undergoes a process having phases that include an establish requirements phase, a high-level design phase, a code phase, a document phase, a unit test phase, and debug phase. The debug phase may include performing iterations of coding, unit testing and integration testing in a process known as debugging. Often, a division of labor is present among employees and contractors of commercial software producers, which permit aspects of each phase to be performed in parallel. Examples of these aspects include the planning of each phase, wherein managers may plan, for example, the code phase simultaneously with planning of the unit test phase. Nevertheless, the actual execution of each phase tends to be linear.
As a development project progresses, greater amounts of labor and capital are invested in the unreleased software without any payment for orders being made. Any delays in phases prior to the unit test phase tend to place pressure on the unit test phase phase. This situation may lead to inadequate testing or delays in making a generally available release. Integration testing tends to be labor intensive because testers verify operation of the software in varying hardware environments and verify bug fixes that are, indeed, fixes. Because integration testing is a part of debugging, testers may test and retest the same component or set of codes as new revisions to the code are available. The ability to eliminate steps and simplify the execution of tests is highly beneficial.
Some strides have been made to automate testing steps. For example, test managers have developed workflows, which permit automated installation of both applications and operating systems onto varying hardware configurations. These workflows include, for example, scripts to identify hardware, allocate hardware, and configure hardware with installed software and data.
Test managers deliver testbuckets or test suites, when they engage in test planning. Testbuckets are a collection of test cases that testers run to verify the expectations of a software version's performance. Such testbucket running may assure that under the most frequent conditions expected to occur in the field, the software version is reliable. Often, such tests may automate or simulate inputs that could be expected to come from humans. Some examples of inputs include mouse movements, key-strokes, telemetry, streaming data, and aggregations of such data. When run, a testbucket collects output in one or more files known as test results. Test results can be voluminous. Results are often very repetitive, and long intervals can occur between results that are considered interesting. Interesting results are those that show a flaw in the code under test.
Commercial software development, even for personal computers, account for interdependencies between code and hardware of several different manufacturers. Often, varying drivers are present that are available from manufacturers of monitors and printers. New software often is required to operate with the varying device drivers. Each of these device drivers may have dozens of potential versions that may be encountered in field use of the code under test.
Thus, it would be advantageous to shift the labor involved in executing tests over to a planning stage such that the iterative executing of tests does not inflate a release schedule, while providing for broad test coverage. In other words, it would be advantageous to define as much of the physical configuration of available resources before execution of the actual tests, and assure that such a definition is so rigorous that a machine may perform the steps of provisioning the system-under-test.